{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ffd5516d-e47a-42c0-988a-6d72874336a1",
   "metadata": {},
   "source": [
    "# Lesson 1: Exercise 1\n",
    "\n",
    "## Creating Data and Backtesting a Trading Strategy\n",
    "\n",
    "---\n",
    "\n",
    "## Overview\n",
    "\n",
    "Welcome to this Excercise. We are now going to put our skills to the test and create a dataset and prepare our first backtests. Proper data preparation is essential to conducting succesful backtest, and we want to make sure that we build a solid foundation and develop proper techniques to ensure that we can build our datasets easily at scale and develop proper data wrangling and preparation approaches. In particular, we are going to cover the  essential steps that are necessary to succesfully build trading  models beginning from creating datasets from publicly available stock price data to conducting exploratory data analysis to finally building and testing models. We are then quickly shifts gears and illustrate a few advanced investment ideas that can be used for further analysis.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6c7e573",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install lxml==6.0.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "85a136b7-fa22-4542-827c-047ff2c4d97a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "\n",
    "# Manually set the path relative to the py file's location that you want to import\n",
    "func_lib_path = os.path.abspath(os.path.join(os.getcwd(), '../../'))\n",
    "# Add the path to sys.path\n",
    "sys.path.append(func_lib_path)\n",
    "# Now you can import func_lib\n",
    "import func_lib\n",
    "\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "# Calculate accuracy, precision, and recall\n",
    "from sklearn.metrics import accuracy_score, precision_score, recall_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "0f7bb375-1bbf-4e6e-b035-dceca5cb29f3",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[*********************100%***********************]  500 of 501 completed\n",
      "\n",
      "32 Failed downloads:\n",
      "['GEHC', 'DOW', 'GEV', 'OTIS', 'VST', 'SW', 'FOX', 'MRNA', 'FTV', 'FOXA', 'VICI', 'PLTR', 'IR', 'DELL', 'HPE', 'INVH', 'DAY', 'PYPL', 'ABNB', 'VLTO', 'QRVO', 'LW', 'GDDY', 'SOLV', 'CEG', 'CRWD', 'CTVA', 'CARR', 'UBER', 'KHC', 'KVUE']: YFPricesMissingError('$%ticker%: possibly delisted; no price data found  (1d 2005-01-01 -> 2015-01-01) (Yahoo error = \"Data doesn\\'t exist for startDate = 1104555600, endDate = 1420088400\")')\n",
      "['HWM']: ChunkedEncodingError(ProtocolError(\"Connection broken: InvalidChunkLength(got length b'', 0 bytes read)\", InvalidChunkLength(got length b'', 0 bytes read)))\n"
     ]
    }
   ],
   "source": [
    "#  Create a variable named 'historical_prices' to store historical price data between the dates'2005-01-01' and '2015-01-01'. Use the function 'createHistPrices()' from 'func_lib' to generate or retrieve the historical prices.\n",
    "historical_prices = func_lib.create_hist_prices(start_date = '2005-01-01', end_date = '2015-01-01')\n",
    "# Create a list named 'list_of_momentums' and add 1 to the list\n",
    "list_of_momentums = [1]\n",
    "# Compute the total returns using the historical prices and the list of momentums. Use the 'computingReturns()' function from 'func_lib'. Pass 'historical_prices' and 'list_of_momentums' as arguments to 'func_lib.computingReturns()'.\n",
    "# - Store the result in a variable named 'total_returns'.\n",
    "total_returns     = func_lib.compute_returns(historical_prices, list_of_momentums)\n",
    "# Use the 'dropna()' method on the 'total_returns' DataFrame.\n",
    "total_returns.dropna(inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "eba45154-563d-4866-9f0f-1c9e8974f1c9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>F_1_d_returns</th>\n",
       "      <th>1_d_returns</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ticker</th>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">A</th>\n",
       "      <th>2005-01-04 00:00:00+00:00</th>\n",
       "      <td>-0.000430</td>\n",
       "      <td>-0.026382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-05 00:00:00+00:00</th>\n",
       "      <td>-0.021945</td>\n",
       "      <td>-0.000430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-06 00:00:00+00:00</th>\n",
       "      <td>-0.000880</td>\n",
       "      <td>-0.021945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-07 00:00:00+00:00</th>\n",
       "      <td>-0.004844</td>\n",
       "      <td>-0.000880</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-10 00:00:00+00:00</th>\n",
       "      <td>-0.017257</td>\n",
       "      <td>-0.004844</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  F_1_d_returns  1_d_returns\n",
       "Ticker Date                                                 \n",
       "A      2005-01-04 00:00:00+00:00      -0.000430    -0.026382\n",
       "       2005-01-05 00:00:00+00:00      -0.021945    -0.000430\n",
       "       2005-01-06 00:00:00+00:00      -0.000880    -0.021945\n",
       "       2005-01-07 00:00:00+00:00      -0.004844    -0.000880\n",
       "       2005-01-10 00:00:00+00:00      -0.017257    -0.004844"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the first 5 lines of total returns\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "55b082b4-d2e2-4e16-a788-e89c0b1274a0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 50076 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The CAGR is: 16.84%\n",
      "Sharpe Ratio of Strategy: 0.8\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 19300 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compute cumulative and calendar returns from the total returns data. \n",
    "# Use the 'compute_BM_Perf()' function from 'func_lib' to perform this calculation. \n",
    "# Pass 'total_returns' as an argument to 'func_lib.compute_BM_Perf()'.\n",
    "# - The function returns two outputs: 'cum_returns' for cumulative returns and 'calendar_returns' for calendar returns.\n",
    "# - Assign the outputs to the variables 'cum_returns' and 'calendar_returns' respectively.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1f3c4485-2e21-4b3b-90f6-0093da6d9d7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the Relative Strength Index (RSI) for each 'Ticker' and add it to the 'total_returns' DataFrame.First, group the 'total_returns' DataFrame by the 'Ticker' column.\n",
    "# For each group, select the '1_d_returns' column and apply a transformation using the 'calculate_rsi' function from 'func_lib'.\n",
    "# - Assign the result to a new column named 'RSI' in the 'total_returns' DataFrame.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8fb7ad03-3f23-42d6-ae0c-6a17761a4e34",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>F_1_d_returns</th>\n",
       "      <th>1_d_returns</th>\n",
       "      <th>RSI</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ticker</th>\n",
       "      <th>Date</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">A</th>\n",
       "      <th>2005-01-04 00:00:00+00:00</th>\n",
       "      <td>-0.000430</td>\n",
       "      <td>-0.026382</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-05 00:00:00+00:00</th>\n",
       "      <td>-0.021945</td>\n",
       "      <td>-0.000430</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-06 00:00:00+00:00</th>\n",
       "      <td>-0.000880</td>\n",
       "      <td>-0.021945</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-07 00:00:00+00:00</th>\n",
       "      <td>-0.004844</td>\n",
       "      <td>-0.000880</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005-01-10 00:00:00+00:00</th>\n",
       "      <td>-0.017257</td>\n",
       "      <td>-0.004844</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  F_1_d_returns  1_d_returns  RSI\n",
       "Ticker Date                                                      \n",
       "A      2005-01-04 00:00:00+00:00      -0.000430    -0.026382  NaN\n",
       "       2005-01-05 00:00:00+00:00      -0.021945    -0.000430  NaN\n",
       "       2005-01-06 00:00:00+00:00      -0.000880    -0.021945  NaN\n",
       "       2005-01-07 00:00:00+00:00      -0.004844    -0.000880  NaN\n",
       "       2005-01-10 00:00:00+00:00      -0.017257    -0.004844  NaN"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print out the first 5 rows of the total_returns dataframe\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a204fba9-7b5f-41de-aa4e-78595450c0d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#  Define a trading strategy function based on the RSI (Relative Strength Index). Create a function named 'trading_strategy' that accepts one parameter: 'rsi'.\n",
    "# Inside the function, use an if-else statement to implement the trading logic:\n",
    "#     - If 'rsi' is less than 65, return 1 (indicating a decision to go long).\n",
    "#     - Otherwise, return 0 (indicating no position or a different strategy).\n",
    "# - This function will be used later to determine trading actions based on RSI values.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c72eb7a1-0976-4230-bde5-9e2b0e333c96",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a variable named 'model_name' and set it to the string \"RSI\".\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "0a4426f0-3bca-45e5-a17e-b344da799258",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 60104 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The CAGR is: 15.89%\n",
      "Sharpe Ratio of Strategy: 0.78\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 21268 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compute cumulative and calendar returns for the given trading strategy.\n",
    "# Use the 'compute_strat_perf()' function from 'func_lib' to perform this calculation.\n",
    "# Pass 'total_returns', 'cum_returns', 'calendar_returns', 'trading_strategy', and 'model_name' as arguments.\n",
    "# - The function returns updated 'cum_returns' and 'calendar_returns' based on the strategy's performance.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1b5fd6d1-67f7-4d54-a277-026f666f966f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the true values (y_test) indicating whether future 1-day returns (F_1_d_returns) are larger or smaller than 0.\n",
    "# Use the 'np.where()' function to evaluate the 'F_1_d_returns' column in the 'total_returns' DataFrame.\n",
    "# Store the result in a new column called 'y_test' in the 'total_returns' DataFrame.\n",
    "\n",
    "# Copy the values from the 'Position' column of 'total_returns' to a new column named 'y_pred'.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "56822855-909b-4363-b9ec-4f0905231bf1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert the 'y_test' column from the 'total_returns' DataFrame to a list using the 'tolist()' method.\n",
    "# Store the result in a variable named 'y_test'.\n",
    "# Similarly, convert the 'y_pred' column to a list and store it in a variable named 'y_pred'.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "8388b7db-30f4-40fa-bbc5-fc37bfaf1b09",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the accuracy, precision, and recall of the trading strategy.\n",
    "# - Use the 'accuracy_score()' function to calculate the accuracy of the predictions.\n",
    "#     - Pass the actual values (y_test) and the predicted values (y_pred) as arguments.\n",
    "#     - Store the result in a variable named 'accuracy'.\n",
    "# - Use the 'precision_score()' function to calculate the precision of the predictions.\n",
    "#     - Similarly, pass 'y_test' and 'y_pred' as arguments.\n",
    "#     - Store the result in a variable named 'precision'.\n",
    "# - Use the 'recall_score()' function to calculate the recall of the predictions.\n",
    "#     - Again, pass 'y_test' and 'y_pred' as arguments.\n",
    "#     - Store the result in a variable named 'recall'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "2ccc3389-9ee9-4fb2-94f4-fecff6c1b20e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a DataFrame named 'metrics_df' using the 'pd.DataFrame()' function.\n",
    "# Define a dictionary with two keys: 'Metric' and 'Value'.\n",
    "#     - The 'Metric' key should have a list of strings: ['Accuracy', 'Precision', 'Recall'].\n",
    "#     - The 'Value' key should have a list of the calculated metrics: [accuracy, precision, recall].\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "fb2cb142-ca67-44f3-a3b3-957b909937f1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Metric</th>\n",
       "      <th>Value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Accuracy</td>\n",
       "      <td>0.511735</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Precision</td>\n",
       "      <td>0.512622</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Recall</td>\n",
       "      <td>0.936336</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Metric     Value\n",
       "0   Accuracy  0.511735\n",
       "1  Precision  0.512622\n",
       "2     Recall  0.936336"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Display the 'metrics_df' DataFrame to see the performance of the trading strategy.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
